The present invention relates to an obstacle detection apparatus and a method for detecting an obstacle such as a preceding vehicle, a parked vehicle, or a pedestrian on a road using cameras on a vehicle.
In order to realize safe driving or automatic driving of a vehicle, a technique to detect an obstacle is necessary. This technique is classified into a utilization of a laser or a supersonic wave, and a utilization of a TV camera. An apparatus using the laser is expensive. In an apparatus using the supersonic wave, resolution of the supersonic wave is low and detection accuracy of the obstacle is correspondingly low. Furthermore, an active sensor using the laser or the supersonic wave cannot recognize a driving lane of the vehicle. On the contrary, TV cameras are relatively inexpensive and have suitable resolution, instrumentation accuracy, and instrumentation limit. Furthermore, the driving lane is also recognized. In case of utilizing a TV camera, a method using one TV camera and a method using a plurality of TV cameras (stereo camera) are selectively used. In the method using one TV camera, a road area and an obstacle area are separated from one image taken by the one TV camera according to intensity, color, or texture information. For example, an intensity area of the average level whose brightness degree is low, i.e., a gray area, is extracted from the image as the road area. An area not including texture may be extracted from the image as the road area. The other area in the image is regarded as the obstacle area. However, many obstacles include intensity, color, or texture similar to the road. Accordingly, in this method, it is difficult to separate the obstacle area from the road area.
On the other hand, in the method using a plurality of cameras, the obstacle is detected based on three-dimensional information. In general, this method is called xe2x80x9cstereo viewxe2x80x9d. For example, two cameras are located at the left side and the right side; the same point in three-dimensional space is corresponded between the left image and the right image; and a three-dimensional position of the point is calculated by triangulation. If a position and a posture of each camera for a road plane are previously calculated, the height of an arbitrary point from the road plane in the image is calculated by the stereo view. In this case, the obstacle area is separated from the road area by the height. In the stereo view, a problem in case of utilizing one camera is avoided.
However, in regular stereo view, a problem such as search of correspondence points is included. In general, the stereo view is a technique to calculate a three-dimensional position of arbitrary point of the image in coordinate system fixed to the stereo camera (Hereinafter, it is called xe2x80x9cstereo camera coordinate systemxe2x80x9d). The search for correspondence points represents search calculation necessary for correspondence of the same point in space between the left image and the right image. The calculation burden is extremely high. The search for correspondence points is a factor preventing realization of the stereo view.
In this case, if the road area and the obstacle area are simply separated in the image, the search for correspondence points is not always necessary. For example, the height of an area from the road plane is decided as follows. Assume that a projection point of the same point on the road plane to the left image and the right image is respectively (u, v), (uxe2x80x2, vxe2x80x2). The following equation (1) is concluded.                                           u            xe2x80x2                    =                                    h11u              +              h12v              +              h13                                      h31u              +              h32v              +              h33                                      ,                  xe2x80x83                ⁢                              v            xe2x80x2                    =                                    h21u              +              h22v              +              h23                                      h31u              +              h32v              +              h33                                                          (        1        )            
{right arrow over (h)}=(h11, h12, h13, h21, h22, h23, h31, h32, h33): parameter of the camera dependent on a position and a posture on the road plane, a focal distance of lens of the camera, and the origin of the image.
xe2x80x9c{right arrow over (h)}xe2x80x9d is previously calculated by projection points (ui, vi), (uixe2x80x2, vixe2x80x2) (i=1, 2, . . . , N) of at least four points on the road plane to the left image and the right image. By using the equation (1), in case that arbitrary point P (u, v) on the left image exists on the road plane, a correspondence point Pxe2x80x2 (uxe2x80x2, vxe2x80x2) on the right image is calculated. If the point P exists on the road plane, the points P and Pxe2x80x2 are a correct pair of correspondence points and intensities of the two points are equal. Accordingly, if an intensity of the point P is not equal to an intensity of the point Pxe2x80x2, the point P is decided to belong to the obstacle. In this method, the height of the arbitrary point in the image from the road plane is directly decided using the equation (1). Coefficients in the equation (1) are calculated by projection points of at least four feature points on the road to the left and right images. Accordingly, the search for correspondence points between the left and right images is not necessary.
If the vehicle is moving at low speed on a flat floor in an indoor environment, {right arrow over (h)} is fixed. Therefore, the obstacle is correctly detected using {right arrow over (h)} calculated once. However, if the vehicle is moving outdoors, relative posture of each camera on the road plane changes regularly because of vibration of the vehicle itself and changes in the inclination of the road. As a result, the parameter {right arrow over (h)} of the camera changes in proportion to the movement of the vehicle. Therefore, if the parameter {right arrow over (h)} calculated during a still moment of the vehicle is utilized, if the camera image is transformed by the equation (1), and if an obstacle is detected by subtracting the transformed image from another camera image, the detection accuracy extremely falls.
As mentioned-above, the obstacle detection apparatus is classified into a type using a laser or a supersonic wave (active), and another type using TV camera (passive). However, the obstacle detection apparatus using the laser or the supersonic wave is expensive, the instrumentation accuracy is low and the driving lane is not recognized. Furthermore, in the obstacle detection apparatus using a TV camera, a usable environment is restricted, the search for correspondence points whose calculation load is high is necessary, and the vibration of the vehicle while moving and the inclination of the road are not taken into consideration. As a result, the ability extremely falls in the outdoor environment.
It is an object of the present invention to provide an obstacle detection apparatus and a method for correctly detecting an obstacle on a plane even if the plane inclines or a plurality of cameras vibrate in case of detecting the obstacle by the plurality of cameras on a vehicle.
According to the present invention, there is provided an obstacle detection apparatus, comprising: a plurality of image input units configured to respectively input an image from at least two camera positions, said plurality of image input units being loaded on a moving object; an image memory configured to store a plurality of images input by said plurality of image input units; an image transformation unit configured to transform one image input by first image input unit using each of a plurality of transformation parameters each representing a geometrical relationship among a predetermined plane, the first, and a second image input units, and to generate a plurality of transformed images from a camera position of the second image input unit, each of the plurality of transformed images corresponding to each of the plurality of transformation parameters; a matching processing unit configured to compare each of the plurality of transformed images with another image input by the second image input unit for each area consisting of pixels, and to calculate a coincidence degree of each area between each transformed image and another image; and an obstacle detection unit configured to detect an obstacle area consisting of areas each of which coincidence degree is below a threshold from another image.
Further in accordance with the present invention, there is also provided an obstacle detection method, comprising the steps of: respectively inputting an image from at least two camera positions through a plurality of cameras loaded on a moving object; storing a plurality of input images; transforming one image input by a first camera using each of a plurality of transformation parameters each representing a geometrical relationship among a predetermined plane, the first camera, and a second camera; generating a plurality of transformed images from a camera position of the second camera; comparing each of the plurality of transformed images with another image input by the second camera for each area consisting of pixels; calculating a coincidence degree of each area between each transformed image and another image; and detecting an obstacle area consisting of areas each of which coincidence degree is below a threshold from another image.
Further in accordance with the present invention, there is also provided a computer-readable memory containing computer-readable instructions, comprising: an instruction unit to respectively input an image from at least two camera position through a plurality of cameras loaded on a moving object; an instruction unit to store a plurality of input images; an instruction unit to transform one image input by a first camera using each of a plurality of transformation parameters each representing a geometrical relationship among a predetermined plane, the first camera, and a second camera; an instruction unit to generate a plurality of transformed images from a camera position of the second camera; an instruction unit to compare each of the plurality of transformed images with another image input by the second camera for each area consisting of pixels; an instruction unit to calculate a coincidence degree of each area between each transformed image and another image; and an instruction unit to detect an obstacle area consisting of areas each of which coincidence degree is below a threshold from another image.